1. Field of the Invention
The present invention relates to apparatus and methods for determining the amount of skew between master clock pulses in system environments wherein reliable system operation demands arrival of such clock pulses at a plurality of locations within a predetermined minimum displacement from each other. More particularly, the present invention relates to systems and methods for dynamically identifying the amount of clock pulse skew associated with circuit elements in their operating environment. The present invention is especially useful in high speed computer systems using a multiplicity of processors and interconnected units.
2. Description of the Prior Art
In the multiple processor computer systems environment, clock pulses from a common source are distributed for controlling many widely separated circuit modules. The time delay associated with passage of a signal in the form of an electrical and/or optical pulse through many discrete components and paths are not uniform so that they arrive in skewed time relation to each other. Typically the master clock pulses from a common source are distributed after a multiplicity of switching functions, amplifications, replications, or other signal handling operations so as to allow further distribution of those pulses throughout a complex system in a carefully controlled manner.
System efficiency demands distribution of the timing pulses originating from the master clock pulse source with minimal skew for controlling a variety of electronic data handling and/or computer functions. The clock pulses must arrive at a plurality of distribution points within an extremely close time tolerance and with well preserved signal integrity as compared to the original master clock pulses.
It is possible to minimize a limited amount of clock pulse skew by applying careful attention to the layout and design of the circuit topography. Examples are shown in U.S. Pat. Nos. 4,514,749 by Skoji and 4,926,066 by Maimi et al. These steps are only effective for the chips themselves and cannot address skew from various divergent clock pulse path interconnections. In addition, such skew compensations, once implemented, cannot accommodate introduction of subsequent increments of skew as from component aging, operating environment variations, and so forth.
Optical signal handling components tend to introduce little skew. Thus others have suggested minimizing skew by liberal usage of optical path elements as in copending and commonly assigned U.S. patent application Ser. No. 07/536,270 as well as in U.S. Pat. No. 4,959,540 by Fan et al.
A variety of electronically oriented configurations for minimizing skew are also known. U.S. Pat. No. 4,860,322 by Lloyd employs cross coupling of clock pulses between sets of distribution tree elements which is impractical and unwieldy for large systems. U.S. Pat. No. 4,696,019 by Tulpule et al synchronizes clock pulses between multiple processors by coupling them in a master/slave configuration with majority voting to select the controlling clock pulses.
Measurement of skew dynamically between circuit elements is possible by implanting probes within the hardware. Theoretically it is possible to sense these direct measurements and take steps to adjust the skew periodically or during system operation. However, in high speed, multiprocessor environments, physical complexity renders this option unacceptable.
One solution is described in the above mentioned copending and commonly assigned U.S. patent application Ser. No. 07/536,270 entitled CLOCK DISTRIBUTION APPARATUS AND PROCESSES PARTICULARLY USEFUL IN MULTIPROCESSOR SYSTEMS by Priest et al wherein the total delays of all relevant clock paths is determined. Various delay segments are thereafter incorporated into each path so that the total delay is the same or at least within tolerance for all paths. That is, once the amount of delay needed for each individual clock path is determined to establish a common delay throughout all clock paths, appropriate delay elements are incorporated in each path such as by introducing settings to selectable delay elements. This procedure is effective but cannot accommodate variations in the system components due to differing environments, aging, and the like.
A variety of configurations of selectable delay elements for incorporation in-line with each path have been around for some time including arrangements using both coarse delay elements and fine delay elements. Banks of such parallel elements with the banks coupled in series are shown in U.S. Pat. No. 4,805,195 by Keegan. A dynamic phase comparison to select the appropriate clock pulse delay by means of a multiplexer is described in U.S. Pat. No. 4,755,704 by Flora et al. Finally U.S. Pat. No. 4,833,695 by Greub delays clock pulses in response to an undisclosed external data source with phase comparator circuits coupled to monitor and stabilize operation of the delay circuit elements.
The various prior art devices and systems either are unacceptable for multiple processor environments or significantly increase the cost of such systems while reducing the reliability thereof. None of the known prior art systems are capable of dynamically determining the actual delay associated with the distribution paths of the clock pulses so as to permit regular adjustment of the clock skew with minimal additional circuit elements added to the already complex system configuration.